Search filtered file system using secondary storage, including multi-dimensional indexing and searching of archived files

ABSTRACT

Techniques for enabling user search of content stored in a file archive include providing a search interface comprising a search rules portion and an action rules portion, receiving a file archive search criterion comprising at least one search rule, and searching the file archive using the search criterion. The techniques also include generating a set of files filtered using the search criterion and performing an action specified in the action rules portion on a file included in the set of files.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present document claims the benefit of priority under 35 U.S.C.119(e) from U.S. Provisional Patent 61/617,891, entitled “SEARCHFILTERED FILE SYSTEM USING SECONDARY STORAGE,” filed on Mar. 30, 2012,incorporated by reference herein. The present application is aContinuation of U.S. patent application Ser. No. 13/792,109, entitled“SEARCH FILTERED FILE SYSTEM USING SECONDARY STORAGE, INCLUDINGMULTI-DIMENSIONAL INDEXING AND SEARCHING OF ARCHIVED FILES,” filed onMar. 10, 2013, and which is incorporated by reference in its entiretyherein.

BACKGROUND

A primary copy of data is generally a production copy or other “live”version of the data which is used by a software application and isgenerally in the native format of that application. Primary copy datamay be maintained in a local memory or other high-speed storage devicethat allows for relatively fast data access if necessary. Such primarycopy data is typically intended for short term retention (e.g., severalhours or days) before some or all of the data is stored as one or moresecondary copies, for example, to prevent loss of data in the event aproblem occurred with the data stored in primary storage.

To protect primary copy data or for other purposes, such as regulatorycompliance, secondary copies (alternatively referred to as “dataprotection copies”) can be made. Examples of secondary copies include abackup copy, a snapshot copy, a hierarchical storage management (“HSM”)copy, an archive copy, and other types of copies.

A backup copy is generally a point-in-time copy of the primary copy datastored in a backup format as opposed to in native application format.For example, a backup copy may be stored in a backup format that isoptimized for compression and efficient long-term storage. Backup copiesgenerally have relatively long retention periods and may be stored onmedia with slower retrieval times than other types of secondary copiesand media. In some cases, backup copies may be stored at an offsitelocation.

After an initial, full backup of a data set is performed, periodic,intermittent, or continuous incremental backup operations may besubsequently performed on the data set. Each incremental backupoperation copies only the primary copy data that has changed since thelast full or incremental backup of the data set was performed. In thisway, even if the entire set of primary copy data that is backed up islarge, the amount of data that must be transferred during eachincremental backup operation may be significantly smaller, since onlythe changed data needs to be transferred to secondary storage. Combined,one or more full backup and subsequent incremental copies may beutilized together to periodically or intermittently create a syntheticfull backup copy. More details regarding synthetic storage operationsare found in commonly-assigned U.S. patent application Ser. No.12/510,059, entitled “Snapshot Storage and Management System withIndexing and User Interface,” filed Jul. 27, 2009, now U.S. Pat. No.7,873,806, which is hereby incorporated by reference herein in itsentirety.

An archive copy is generally a copy of the primary copy data, buttypically includes only a subset of the primary copy data that meetscertain criteria and is usually stored in a format other than the nativeapplication format. For example, an archive copy might include only thatdata from the primary copy that is larger than a given size threshold orolder than a given age threshold and that is stored in a backup format.Often, archive data is removed from the primary copy, and a stub isstored in the primary copy to indicate its new location. When a userrequests access to the archive data that has been removed or migrated,systems use the stub to locate the data and often make recovery of thedata appear transparent, even though the archive data may be stored at alocation different from the remaining primary copy data.

Archive copies are typically created and tracked independently of othersecondary copies, such as other backup copies. For example, to create abackup copy, the data storage system transfers a secondary copy ofprimary copy data to secondary storage and tracks the backup copy usinga backup index separate from the archive index. To create an archivecopy, a conventional data storage system transfers the primary copy datato be archived to secondary storage to create an archive copy, replacesthe primary copy data with a stub, and tracks the archive copy using anarchive index. Accordingly, the data storage system will transfer twoseparate times to secondary storage a primary copy data object that isboth archived and backed-up.

Users often need to access files in a secondary or a backup storage witha specific search context in their mind. For example, a user may need toaccess photo files from his last Hawaii trip, archived on a storagesystem, or all documents that include the word “taxes” and so on. Such acontext-sensitive search is cumbersome using presently availabletechniques in which a user has to speculatively mount archived filefolders to the user's computer and then sift through all files in themounted drive to look for files of interest.

In other operational scenarios, a user may want to access a specificportion of an archived media file, such as a home video. Alternatively,a user may want to access the archived home video starting at a specificpoint in the video file. A user may experience long delays in fulfillingsuch requests using conventional techniques in which mounting of thevideo file may take a significant amount of time. Furthermore, such useractivities may tie up valuable computational resources needed formounting files from archives to a local memory and transferring databetween the user device and a secondary storage location.

The need exists for systems and methods that overcome the aboveproblems, as well as systems and methods that provide additionalbenefits. Overall, the examples herein of some prior or related systemsand methods and their associated limitations are intended to beillustrative and not exclusive. Other limitations of existing or priorsystems and methods will become apparent to those of skill in the artupon reading the following Detailed Description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of suitableinformation management environment in which aspects of the inventivesystem may operate.

FIG. 2 illustrates examples of production and non-production data thatmay be processed by the information management environment of FIG. 1.

FIG. 3 is a block diagram illustrating one example of a hierarchicalarrangement of resources for the information management system of FIG.1, including information management cells.

FIG. 4 is a block diagram illustrating an example of resources formingan information management cell.

FIG. 5A is an architectural block diagram of a suitable computernetwork.

FIG. 5B is a block diagram depiction of the operation of a SearchFiltered File System.

FIG. 6 is a flow chart representation of a multi-dimensional indexingprocess.

FIG. 7 is an example data structure used for storage of indexed archivefiles in a secondary storage device.

FIG. 8 is an example menu screen available to a user for criteria-basedsearch of an archive.

FIG. 9 is a flow chart representation of a process of providing filesearch results to a user.

FIG. 10 is a flow chart representation of a process of providing mediacontent search results to a user.

DETAILED DESCRIPTION

The techniques disclosed in this document are useful, in one aspect, insolving the above-discussed problems related to searching for and withinfiles stored in an archived file folder. In another aspect, thetechniques are useful in providing quick response to a user's requeststo view a portion of media content from an archived media file such as adigital video or audio file.

The headings provided herein are for convenience only and do notnecessarily affect the scope or meaning of the disclosure.

Overview

A software, firmware, and/or hardware system is disclosed that, amongother things, performs a multi-dimensional indexing of files duringarchival of the files to a secondary storage. The results of theindexing are stored at a network location that may be separate from orthe same as the location of storage of the archive files. Furthermore,techniques are disclosed that allow a user to search for a file orcontent within a file using a specific search criterion. In one aspect,the search experience is simplified for a user by mounting only thefiles that meet the search criteria to a local copy of the archive. Inanother aspect, media files, which often are relatively large in size,are staged or streamed in smaller pieces from the archive copy, based onsearch parameters provided by the user.

In one implementation, a network file system (NFS) server is used toprovide an NFS service to a user. The NFS service provides an interfaceto an underlying index (e.g., a c-tree index) that represents thecontents of secondary copies of media files, such as backup copies ordeduplicated, archive copies of media files. A user can use a standardNFS client to connect to this NFS service and browse a directorystructure of the secondary copies of media files and seek or “fastforward” to random offsets within a secondary copy of a media file,without needing to restore the entire media file.

The NFS service allows files to be seen and accessed via differentclient devices. The NFS service acts as an interface to secondary files,such as archived media files. Under one implementation, the user canmount a shared file system or a drive, which fits the user's searchcriteria, exported by the NFS service to his NFS client device. A clientdevice sends an rpc (remote procedure call) to access files and foldersthat match the search criteria. The NFS service processes the RPC byconsulting with the index to identify the location of the desired data,such as a media or video file stored in secondary storage. The systemmay then retrieve a portion (as desired by the client) of the video filefor delivery to the requesting client.

If the video file is provided, the client may issue a seek command to“fast forward” to another point in the video. In response, the systemagain traps the seek command at the NFS. The system determines anappropriate offset from a start of the video (e.g. determines a jump to⅔ into the video from a GUI of a video/media player) and determines aproportional or estimated offset of the video file. If the secondarycopy of the video includes a data structure or file system indicating anumber of blocks for the video file, then the system can jump to a blockapproximately ⅔ of the total number of blocks for the video file.

Various examples of the techniques will now be described. The followingdescription provides specific details for a thorough understanding andenabling description of these examples. One skilled in the relevant artwill understand, however, that the invention may be practiced withoutmany of these details. Likewise, one skilled in the relevant art willalso understand that the invention may include many other obviousfeatures not described in detail herein. Additionally, some well-knownstructures or functions may not be shown or described in detail below,so as to avoid unnecessarily obscuring the relevant description.

The terminology used below is to be interpreted in its broadestreasonable manner, even though it is being used in conjunction with adetailed description of certain specific examples of the invention.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in this Detailed Descriptionsection.

Information Management Environment

Aspects of the technologies described herein may be practiced in aninformation management environment 100, which will now be describedwhile referencing FIG. 1. As shown in FIG. 1, the environment 100includes multiple computing devices that execute numerous softwareapplications to facilitate the operations of an organization (ormultiple affiliated organizations), such as a household, corporation orother business entity, a non-profit organization, an educationalinstitution, or a governmental agency. The computing devices may includeone or more: servers 105 (such as mail servers, file servers, databaseservers, print servers, and web servers), personal computers 110,workstations 115, or other types of fixed computing systems such asmainframe computers and minicomputers (not shown). The servers 105 mayinclude network-attached storage (NAS) filers.

The environment 100 may include virtualized computing resources, such asa virtual machine 120 provided to the organization by a third-partycloud service vendor or a virtual machine 125 running on a virtualmachine host 130 operated by the organization. For example, theorganization may use one virtual machine 125A as a database server andanother virtual machine 125B as a mail server. The environment 100 mayalso include mobile or portable computing devices, such as laptops 135,tablet computers 140, personal data assistants 145, mobile phones 152(such as smartphones), and other mobile or portable computing devicessuch as embedded computers, set top boxes, vehicle-mounted devices,wearable computers, etc.

Of course, other types of computing devices may form part of theenvironment 100. As part of their function, each of these computingdevices creates, accesses, modifies, writes, and otherwise usesproduction copies of data and metadata that are typically stored in apersistent storage medium having fast I/O times. For example, eachcomputing device may regularly access and modify data files and metadatastored on semiconductor memory, a local disk drive or a network-attachedstorage device. Each of these computing devices may access data andmetadata via a file system supported by an operating system of thecomputing device.

The environment 100 may also include hosted services 122 that providevarious online services to the organization or its constituent members(e.g., the organization's departments, employees, independentcontractors, etc.) such as social networking services (e.g., Facebook,Twitter, Pinterest), hosted email services (e.g., Gmail, Yahoo Mail,Hotmail), or hosted productivity applications or other hostedapplications (e.g., Microsoft Office 365, Google Docs, Salesforce.com).Hosted services may include software-as-a-service (SaaS),platform-as-a-service (PaaS), application service providers (ASPs),cloud services, and all manner of delivering computing or functionalityvia a network. As it provides services to users, each hosted service maygenerate additional “hosted data and metadata” that is associated witheach user. For example, Facebook may generate and store photos, wallposts, notes, videos, and other content that are associated with aparticular Facebook user's account.

The organization directly or indirectly employs an informationmanagement system 150 to protect and manage the data and metadata usedby the various computing devices in the environment 100 and the data andmetadata that is maintained by hosted services on behalf of usersassociated with the organization. One example of an informationmanagement system is the CommVault Simpana® software system, availablefrom CommVault Systems, Inc. of Oceanport, N.J. The informationmanagement system creates and manages non-production copies of the dataand metadata to meet information management goals, such as: permittingthe organization to restore data, metadata or both data and metadata ifan original copy of the data/metadata is lost (e.g., by deletion,corruption, or disaster, or because of a service interruption by ahosted service); allowing data to be recovered from a previous time;complying with regulatory data retention and electronic discovery(“e-discovery”) requirements; reducing the amount of data storage mediaused; facilitating data organization and search; improving user accessto data files across multiple computing devices and/or hosted services;and implementing information lifecycle management (“ILM”) or other dataretention policies for the organization. The information managementsystem 150 may create the additional non-production copies of the dataand metadata on any suitable non-production storage medium such asmagnetic disks 155, magnetic tapes 160, other storage media 165 such assolid-state storage devices or optical disks, or on cloud data storagesites 170 (e.g. those operated by third-party vendors). Further detailson the information management system may be found in the assignee's U.S.patent application Ser. No. 12/751,850, filed Mar. 31, 2010 entitledDATA OBJECT STORE AND SERVER FOR A CLOUD STORAGE ENVIRONMENT, INCLUDINGDATA DEDUPLICATION AND DATA MANAGEMENT ACROSS MULTIPLE CLOUD STORAGESITES, now U.S. Pat. No. 8,285,681 (attorney docket 060692-8075.US02),which is hereby incorporated herein by reference in its entirety.

FIG. 2 helps illustrates some differences between “production copies” ofdata and metadata and “non-production copies” of data and metadata inthe data management environment 100. As shown, each computing device 205in the environment 100 has at least one operating system 210 installedand one or more applications 215A-D, such as mail server applications,file server applications, mail client applications, databaseapplications, word processing applications, spreadsheet applications,presentation applications, browser applications, mobile applications,entertainment applications, and so on. Each application can access andmodify various production copies of files stored in a production datastorage medium 218, which may be a network attached storage filer orform part of a Hadoop distributed file system, Open VMS file system, orother type of distributed file system, simply by making conventionalfile system calls via the operating system 210, without needing theinformation management system 150 to intervene on behalf of theoperating system or application. The production copies of files mayinclude structured data (e.g., database files), unstructured data (e.g.,documents), and/or semi-structured data, and thus may include one ormore documents 220A-B, spreadsheets 225, presentation documents 230,video files 232, image files 234, email mailboxes 235, html/xml or othertypes of markup language files 222, and/or databases 240. The operatingsystem 210 may also access and modify production copies of files andother data, such as files in a system volume and/or boot volume. Thehosted data and metadata used by a hosted service are also “productioncopies” of data and metadata since the hosted service accesses andmodifies the user's data and metadata as part of its services.Production copies of data may include not only files, but also subsetsof files, each of which a related application 215 or the operatingsystem 210 treats as an independent functional unit, but which is notseparately addressed in the associated file system. For example, asingle email mailbox 235 may include multiple email messages 245A-C,email headers, and attachments. As another example, a single database240 may include multiple tables 255A-C. As used herein, a “data object”refers to both (1) any file that is currently or previously addressableby a file system and (2) a functional subset of such a file that has aparticular function for a related application 215A-D or the operatingsystem 210. Each data object may be further decomposed into one or moredata blocks each of which is a collection of data bits within the dataobject that may not have any particular function for a relatedapplication or the operating system. In addition to data objects, theoperating system 210 and applications 215A-D may also access and modifyproduction copies of metadata, such as boot sectors, partition layouts,file or data object metadata (e.g., file name, file size,creation/modification/access timestamps, file location within a filefolder directory structure, user permissions, owners, groups, accesscontrol lists (“ACLs”)), and system metadata (e.g., registryinformation). In addition to metadata generated by or related to filesystems and operating systems, some applications maintain indices ofproduction metadata for data objects, e.g., metadata associated withindividual email messages. Thus, as shown in FIG. 2, each data objectmay be associated with a production copy of object metadata(“Meta1-11”), which may be file system metadata and/orapplication-specific metadata.

The information management system 150 accesses or receives copies of thevarious production copies of data objects and metadata, and via aninformation management operation (such as a backup operation, archiveoperation, or snapshot operation), creates non-production copies ofthese data objects and metadata, often stored in one or morenon-production storage mediums 265 different than the production storagemedium 218 where the production copies of the data objects and metadatareside. A non-production copy of a data object represents the productiondata object and its associated metadata at a particular point in time(non-production objects 260A-C). Since a production copy of a dataobject or metadata changes over time as it is modified by an application215, hosted service 122, or the operating system 210, the informationmanagement system 150 may create and manage multiple non-productioncopies of a particular data object or metadata, each representing thestate of the production data object or metadata at a particular point intime. Moreover, since a production copy of a data object may eventuallybe deleted from the production data storage medium and the file systemfrom which it originated, the information management system may continueto manage point-in-time representations of that data object, even thougha production copy of the data object itself no longer exists.

For virtualized computing devices, such as virtual machines, theoperating system 210 and applications 215A-D may be running on top ofvirtualization software, and the production data storage medium 218 maybe a virtual disk created on a physical medium such as a physical disk.The information management system may create non-production copies ofthe discrete data objects stored in a virtual disk file (e.g.,documents, email mailboxes, and spreadsheets) and/or non-productioncopies of the entire virtual disk file itself (e.g., a non-productioncopy of an entire .vmdk file).

Each non-production object 260A-C may contain copies of or otherwiserepresent more than one production data object. For example,non-production object 260A represents three separate production dataobjects 255C, 230 and 245C (represented as 245C′, 230′ and 245′,respectively). Moreover, as indicated by the prime mark (′), anon-production object may store a representation of a production dataobject or metadata differently than the original format of the dataobject or metadata, e.g., in a compressed, encrypted, deduplicated, orotherwise optimized format. Although FIG. 2 shows that a singleproduction data object (e.g., 255C), and its associated data objectmetadata (e.g., Meta11) are represented by the contents of only a singlenon-production object (e.g., 260A), the entire contents of a singleproduction data object and/or its metadata at a particular point in timemay instead span across numerous non-production objects. Also a singlenon-production object 260 may contain copies of or otherwise representproduction data objects that originated from different computingdevices.

Non-production copies include backup copies, archive copies, andsnapshot copies. Backup copies are generally used for shorter-term dataprotection and restoration purposes and may be in a native applicationformat or in a non-native format (e.g., compressed, encrypted,deduplicated, and/or otherwise modified from the original applicationformat). Archive copies are generally used for long-term data storagepurposes and may be compressed, encrypted, deduplicated and/or otherwisemodified from the original application format. In some examples, when anarchive copy of a data object is made, a logical reference or stub maybe used to replace the production copy of the data object in theproduction storage medium 218. In such examples, the stub may point toor otherwise reference the archive copy of the data object stored in thenon-production storage medium so that the information management systemcan retrieve the archive copy if needed. The stub may also include somemetadata associated with the data object, so that a file system and/orapplication can provide some information about the data object and/or alimited-functionality version (e.g., a preview) of the data object. Asnapshot copy represents a data object at a particular point in time. Asnapshot copy can be made quickly and without significantly impactingproduction computing resources because large amounts of data need not becopied or moved. A snapshot copy may include a set of pointers derivedfrom the file system or an application, where each pointer points to arespective stored data block, so collectively, the set of pointersreflect the storage location and state of the data object at aparticular point in time when the snapshot copy was created. In“copy-on-write”, if a block of data is to be deleted or changed, thesnapshot process writes the block to a particular data storage location,and the pointer for that block is now directed to that particularlocation. The set of pointers and/or the set of blocks pointed to by asnapshot may be stored within the production data storage medium 218.

Non-production copies of a data object or metadata may be distinguishedfrom a production copy of a data object or metadata in several ways.First, a non-production copy of a data object is created to meet thedifferent information management goals described above and is notdirectly used or modified by applications 215A-D, hosted services 122,or the operating system 210. Second, a non-production copy of a dataobject is stored as one or more non-production objects 260 that may havea format different from the native application format of the productioncopy of the data object, and thus often cannot be directly used by thenative application or a hosted service 122 without first being modified.Third, non-production objects are often stored on a non-productionstorage medium 265 that is inaccessible to the applications 215A-Drunning on computing devices and hosted services 122. Also, somenon-production copies may be “offline copies,” in that they are notreadily available (e.g. not mounted tape or disk.) Offline copiesinclude copies of data that the information management system can accesswithout any human intervention (e.g. tapes within an automated tapelibrary, but not yet mounted in a drive), and copies that theinformation management system 150 can access only with at least somehuman intervention (e.g. tapes located at an offsite storage site).

The information management system 150 also generates informationmanagement data 275, such as indexing information, that permit theinformation management system to perform its various informationmanagement tasks. As shown in FIG. 2, a computing device 205 may includeone or more data management agents 280 that provide client-sidefunctions for the information management system.

Information Management System

FIG. 3 shows a hierarchical arrangement of resources that may form aninformation management system 300. As shown, the information managementsystem 300 includes multiple information management cells 350 arrangedin a hierarchy, with some information management cells (e.g., 350D-E)subordinate to other information management cells (e.g., 350B). A globalstorage manager 305, which may form part of a global information cell350 x, manages the overall hierarchy of information management cells byreceiving reports from the various subordinate information managementcells related to the operation of the cells, determining globalinformation management policies in light of received reports, andpushing information management policies towards subordinate cells forimplementation. The global storage manager may also maintain anddisseminate, to the various cells, system-wide information managementdata. A superior information management cell (e.g., 350B), may performsimilar tasks for its subordinate cells (e.g., 350D-E) and/or otherwiseact as an intermediary between the subordinate cells and the globalstorage manager 305.

Information Management Cell

FIG. 4 shows an arrangement of resources that may form an informationmanagement cell 350. As shown, the information management cell includesa storage manager 402, one or more media agents 410A-M, one or morenon-production storage mediums 155-170, one or more computing devices205A-N, and one or more production data storage mediums 218A-N. Outsideof the information management cell are hosted services 122, which mayinteract with media agents 410 and its components, as described furtherherein. In some examples, all or a portion of an information managementcell may be implemented as an object store, as described in assignee'sU.S. patent application Ser. No. 12/751,850 (introduced above).

The storage manager 402 may be a software module or other applicationthat coordinates and controls information management operationsperformed by one or more information management cells 350 to protect andcontrol copies of non-production data objects and metadata. As shown bythe dashed lines 416 and 422, the storage manager may communicate withsome or all elements of the information management cell 350, such as themedia agents 410 and computing devices 205, to initiate and managebackup operations, snapshot operations, archive operations, datareplication operations, data migrations, data distributions, datarecovery, and other information management operations. The storagemanager may control additional information management operationsincluding ILM, deduplication, content indexing, data classification,data mining or searching, e-discovery management, collaborativesearching, encryption, and compression. Alternatively or additionally, astorage manager may control the creation and management of disasterrecovery copies, which are often created as secondary, high-availabilitydisk copies, using auxiliary copy or replication technologies.

The storage manager 402 may include a jobs agent 455, a management agent450, a network agent 445, and an interface agent 460, all of which maybe implemented as interconnected software modules or applicationprograms. The jobs agent 455 monitors the status of informationmanagement operations previously performed, currently being performed,or scheduled to be performed by the information management cell 350. Themanagement agent 450 provides an interface that allows variousmanagement agents 450 in multiple information management cells 350 (orin a global storage manager 305) to communicate with one another. Thisallows each information management cell 350 to exchange statusinformation, routing information, capacity and utilization information,and information management operation instructions or policies with othercells. In general, the network agent 445 provides the storage manager402 with the ability to communicate with other components within theinformation management cell and the larger information managementsystem, e.g., via proprietary or non-proprietary network protocols andapplication programming interfaces (“APIs”) (including HTTP, HTTPS, FTP,REST, virtualization software APIs, cloud service provider APIs, hostedservice provider APIs). The interface agent 460 includes informationprocessing and display software, such as a graphical user interface(“GUI”), an API, or other interactive interface through which users andsystem processes can retrieve information about the status ofinformation management operations or issue instructions to theinformation management cell and its constituent components. The storagemanager 402 may also track information that permits it to select,designate, or otherwise identify content indices, deduplicationdatabases, or similar databases within its information management cell(or another cell) to be searched in response to certain queries.

The storage manager 402 may also maintain information management data,such as a database 465 of management data and policies. The database 465may include a management index that stores logical associations betweencomponents of the system, user preferences, user profiles (that amongother things, map particular information management users to computingdevices or hosted services), management tasks, or other useful data. Thedatabase 465 may also include various “information management policies,”which are generally data structures or other information sources thateach include a set of criteria and rules associated with performing aninformation management operation. The criteria may be used to determinewhich rules apply to a particular data object, system component, orinformation management operation, an may include:

-   -   frequency with which a production or non-production copy of a        data object or metadata has been or is predicted to be used,        accessed, or modified;    -   access control lists or other security information;    -   the sensitivity (e.g., confidentiality) of a data object as        determined by its content and/or metadata;    -   time-related factors;    -   deduplication information;    -   the computing device, hosted service, computing process, or user        that created, modified, or accessed a production copy of a data        object; and    -   an estimated or historic usage or cost associated with different        components.

The rules may specify, among other things:

-   -   a schedule for performing information management operations,    -   a location (or a class or quality of storage media) for storing        a non-production copy,    -   preferences regarding the encryption, compression, or        deduplication of a non-production copy,    -   resource allocation between different computing devices or other        system components (e.g., bandwidth, storage capacity),    -   whether and how to synchronize or otherwise distribute files or        other data objects across multiple computing devices or hosted        services,    -   network pathways and components to utilize (e.g., to transfer        data) during an information management operation, and    -   retention policies (e.g., the length of time a non-production        copy should be retained in a particular class of storage media).

As noted above, each computing device 205 may include one or more datamanagement agents 280. Each data management agent is a software moduleor component that helps govern communications with other systemcomponents. For example, the data management agent receives commandsfrom the storage manager 402 and sends to and receives from media agents410 copies of data objects, metadata, and other payload (as indicated bythe heavy arrows). Each data management agent accesses data and/ormetadata stored in a production data storage medium 218 and arranges orpacks the data and metadata in a certain format (e.g., backup or archiveformat) before it is transferred to another component. Each datamanagement agent can also restore a production copy of a data object ormetadata in a production data storage medium 218 from a non-productioncopy. A data management agent may perform some functions provided by amedia agent, which are described further herein, such as compression,encryption, or deduplication. Each data management agent may bespecialized for a particular application (e.g. a specified datamanagement agent customized to handle data generated or used by Exchangeby Microsoft Corp.). Alternatively or additionally, a more generic datamanagement agent may handle data generated or used by two or moreapplications.

Each computing device 205 may also include a data distribution and livebrowsing client module 405 (herein “distribution client module”). Thedistribution client module 405 is responsible for, inter alia,associating mobile devices and/or hosted service accounts with users ofthe information management system, setting information managementpolicies for mobile and other computing devices, pushing data objects toa distribution module for distribution to other computing devices,providing unified access to a user's data via an interface, andproviding live browsing features. The various functions of thedistribution client module are described in greater detail herein.

A media agent 410, which may be implemented as a software module,conveys data, as directed by the storage manager 402, between acomputing device 205 (or hosted service 122) and one or morenon-production storage mediums 155-870. Each media agent 410 may controlone or more intermediary storage devices 418, such as a cloud server ora tape or magnetic disk library management system, to read, write, orotherwise manipulate data stored in a non-production storage medium155-870. Each media agent 410 may be considered to be “associated with”a storage device and its related non-production storage media if thatmedia agent is capable of routing data to and storing data in thestorage media managed by the particular storage device. A media agentmay communicate with computing devices 205, hosted services 122, storagedevices 418A-D, and the storage manager 402 via any suitablecommunications path, including SCSI, a Storage Area Network (“SAN”), aFibre Channel communications link, or a wired, wireless, or partiallywired/wireless computer or telecommunications network, including theInternet.

To perform its functions, the media agent 410 may include a media filesystem module 425, a data classification module 435, a content indexingmodule 420, a deduplication module 430, an encryption module 475, acompression module 485, a network module 415, a distribution module 490,and a media agent database 440. The media file system module 425 isresponsible for reading, writing, archiving, copying, migrating,restoring, accessing, moving, sparsifying, deleting, sanitizing,destroying, or otherwise performing file system operations on variousnon-production storage devices of disparate types. The media file systemmodule may also instruct the storage device to use a robotic arm orother retrieval means to load or eject certain storage media such as atape.

The network module 415 permits the media agent to communicate with othercomponents within the system and hosted services 122 via one or moreproprietary and/or non-proprietary network protocols or APIs (includingcloud service provider APIs, virtual machine management APIs, and hostedservice provider APIs). The deduplication module 430 performsdeduplication of data objects and/or data blocks to reduce dataredundancy in the cell. The deduplication module may generate and storedata structures to manage deduplicated data objects, such asdeduplication tables, in the media agent database 440. The encryptionmodule 475 performs encryption of data objects, data blocks, ornon-production objects to ensure data security in the cell. Thecompression module 485 performs compression of data objects, datablocks, or non-production objects to reduce the data capacity needed inthe cell.

The content indexing module 420 analyzes the contents of productioncopies or non-production copies of data objects and/or their associatedmetadata and catalogues the results of this analysis, along with thestorage locations of (or references to) the production or non-productioncopies, in a content index stored within a media agent database 440. Theresults may also be stored elsewhere in the system, e.g., in the storagemanager 402, along with a non-production copy of the data objects,and/or an index cache. Such index data provides the media agent 410 oranother device with an efficient mechanism for locating productioncopies and/or non-production copies of data objects that matchparticular criteria. The index data or other analyses of data objects ormetadata may also be used by the data classification module 435 toassociate data objects with classification identifiers (such asclassification tags) in the media agent database 440 (or other indices)to facilitate information management policies and searches of storeddata objects.

The distribution module 490 may be a set of instructions thatcoordinates the distribution of data objects and indices of dataobjects. The distribution may occur from one computing device 205 toanother computing device 205 and/or from hosted services 122 tocomputing devices 205. As a first example, the distribution module maycollect and manage data and metadata from hosted services 122 or mobiledevices 205. As another example, the distribution module may synchronizedata files or other data objects that are modified on one computingdevice so that the same modified files or objects are available onanother computing device. As yet another example, the distributionmodule may distribute indices of data objects that originated frommultiple computing devices and/or hosted services, so a user can accessall of their data objects through a unified user interface or a nativeapplication on their computing device. The distribution module may alsoinitiate “live browse” sessions to permit communications betweendifferent computing devices so that the devices can interchange data andmetadata or so the devices can provide computing resources, such asapplications, to each other. The functions performed by the distributionmodule are described in greater detail herein.

Suitable Computer Network

FIG. 5A is a block diagram depicting certain functional blocks of asearch file system 500. Files may be transferred to/from a user device502 to an archive or a secondary storage (not shown in FIG. 5A) asfurther described below. When files from the user device 502 arearchived or transferred from a primary storage on the user device 502 toa secondary storage or an archive, a multi-dimensional indexer 504 mayoperate on the files to generate one or more of: a file attribute index508, a content index 510 and a file archive 512, each of which aredescribed herein. The multi-dimensional indexer 504 may be implementedas a part of the previously described media agent or may be executed asa separate application.

The user device 502 may be any computing platform such as a mobilephone, a smartphone, a tablet, a handheld computer, a laptop, a desktop,a server, and so on. No assumptions are made about any specificoperating system being available on the user device 502. The user device502 may be battery operated or may be powered from a power outlet.

In some implementations, the multi-dimensional indexer 504, which isfurther described below, may be implemented on the user device 502. Insome implementations, the multi-dimensional indexer 504 may beimplemented as a part of the media agent (previously discussed) on ahardware platform that is coupled to the user device 502 via a computernetwork. Other implementations are possible in which a portion of theindexing is implemented on the user device 502 and another portion ofthe indexing is implemented on another computer. For example, in oneimplementation, file attribute indexing, which typically needs lowcomputational resources, may be performed on the user device 502, whilecontent indexing, which typically needs higher computational resources,may be performed on another server in the network.

Multi-Dimensional Indexing

The multi-dimensional indexer 504 may in general operate on files from auser device 502 that are being transferred to a secondary storage or anarchive. The transfer of file may be being performed by an explicitcommand from a user or due to an implicit command such as a scheduledbackup operation or a system administrator controlled backup of certainfiles and folders from the user device 502.

In some implementations, the multi-dimensional indexer 504 may perform afile attribute indexing operation on a file being archived. It is wellknown that computer files have attributes such as file name, edit time,author name, last saved, file type, file size, and so on. Severaltechniques, e.g., C-tree based indexing, are well known in the art forindexing files based on file attributes. A description of thesewell-known techniques is omitted from this document for brevity.

In some implementations, the multi-dimensional indexer 504 may performcontent indexing on the file being archived. The content of the file maybe indexed along several “dimensions” or attributes of the content ofthe file. For example, a textual content indexer may identify and indexwords or phrases occurring in the file. A picture content indexer mayidentify types and number of pictures embedded in a document. In someimplementations, the picture content indexer may use pattern recognitiontechniques to extract content information from the images (e.g., faceidentification, natural light/flash photograph, does the image includeflowers, trees, humans, blue sky, etc.). The picture attributes may begenerated on-the-fly or may be generated based on a pre-determined setof picture attributes that the picture content indexer is programmed tolook for. A video content indexer may similarly perform patternrecognition, scene recognition, key frame indexing (e.g., frames thatare self-encoded and can be used for trick modes such as fast forwardingor rewinding), etc. on a video file being archived. An audio contentindexer may perform indexing that uses audio recognition techniques suchas genre detection, musical instrument detection, song recognition basedon tune matching, etc.

The various indices generated by the multi-dimensional indexer 504during the archiving of a file or a folder that contains one or morefiles or folders may be stored in three logical groups of indexedinformation: A file attribute index 508, a content index 510 and thearchived file version 512 (which has been discussed previously). Whilethe indices 508, 510, 512 are described as being different logicalgroups; such grouping is only being made for the sake of clarity ofexplanation. In actual implementations, the logical groups may beimplemented as a single or multiple databases and may be stored on thesame computer or distributed across different platforms in a network.

In some implementations, the file attribute index 508 contains “blackbox” information or metadata about a file that is generally independentof the actual content of the file. This information may include, e.g.,file size, last edit time, last modified by field, creation time, lastprinted time, user or users authorized to read the file, and so on.

In some implementations, the content index 510 includes informationcharacterizing the contents of a file, which is generally independent ofthe “black box” features. For example, as previously described thecontent index may include information about whether content includesaudio, video, images, identification of a topic of the document based ontext keywords, whether the file includes embedded macrocode, and so on.

Criteria Search Interface

When searching for files in archives, a user may be able to access acriteria search interface 506 using his user device 502. In differentimplementations, the criteria search interface 506 may be invoked on thesame platform as the user device 502, or may be invoked on anotherserver, with the user being able to interact with the criteria searchinterface 506 over a network connection. As further described below, thecriteria search interface 506 may allow a user to search a file archiveor a secondary storage (e.g., file archive 512). The search results maybe presented to the user using a standard mechanism such as a NetworkFile System (NFS) service 514 mechanism. In one aspect, a Search Filter516 may make available for user interaction through the NFS service 514,the files and folders from the file archive 512 that meet the user'ssearch criteria.

One beneficial aspect of such a presentation of files to the user may bethat the user is able to view and interact with file listings using afamiliar file and folder interface. Another beneficial aspect may bethat only files that the user may be interested in are displayed in thelisting presented to the user. Another advantageous aspect may be thatthe files made available by the Search Filter 516 may be mounted to adrive using well known NFS mechanism, thereby allowing user interactionsuch as play, open, copy, move, delete, etc. without having to changeuser experience for executing such commands and without having to writeadditional software for implementing such commands. Another beneficialaspect of mounting files (or file clips) that meet the user's searchcriteria using NFS 514 may be that such selective mounting from the FileArchive 512 may produce expedient response from the system because thesystem need not mount the entire File Archive 512 before allowingbrowsable access to a user.

Search Filtered File System

FIG. 5B is a block diagram depicting an example of the operation of asearch filtered file system. In some implementations without the searchfilter 516 described above, the NFS interface 514 that enables a user tosearch through network files may exchange messages 558 with a c-treedatabase 552 that provides the NFS interface 514 with informationregarding files and folders available on stored files 554 in thenetwork. The c-tree database 552 may include information about filesstored in archives or secondary storages 556. However, files stored in atypical file archive are deduplicated and encoded to achieve datacompression (not in a native format), and therefore may not beaccessible via the NFS 514 for user interaction such as open, close,move, copy, etc.

In some implementations, the search filter 516 may operate at a protocollayer between the NFS 514 and the database 552 (e.g., a C-tree database)such that the search filter 516 traps communication between the NFS 514and the database 552. In addition, as previously described, the searchfilter 516 may mount a file system from the archive based on a usersearch of the archive such that the search filtered file system iscopied into a local cache in the non-archived (e.g., non-deduplicatedand non-compressed) format. When the NFS service 514 wishes to access afile that is listed in the search filtered file system, such a seekrequest 560 is trapped by the search filter 516. In response to such acall, the searched file system reports back the search results that werecopied to the local cache from the archive, making these resultsavailable to the NFS service 514 for user interaction.

In some implementations, the search filter 516 may provide a database ofthe search filtered files to the NFS 514. In one aspect, the searchfiltered file may therefore appear to a user as a “normal” network filesystem, while the search filter 516 performing the task of translatingvarious NFS commands into corresponding acts of files in the filearchive. Therefore, tasks such as figuring out where in an archive thegiven file is located, how many blocks (in size) are occupied by thefile and de-archiving the file are coordinated by the search filter 516transparently to the user and the NFS 514.

FIG. 6 is a flow chart description of a procedure 600 of indexing datafiles being archived. As previously described, the procedure 600 may beimplemented on a user's computer, on a media agent platform or acombination of the two.

At 602, a command to begin multi-dimensional indexing of a file isreceived. As discussed previously, the command may be explicitly issuedby a user, or may be implicitly generated based on a pre-scheduledarchival of the file.

At 604, a file attribute index is generated. The file attribute indexincludes information related to extrinsic properties of a file, such asfile size, last edited time, author, authorized users, etc. For example,when the item being indexed is an email, the file attribute index mayinclude information regarding date the email was sent, to whom, fromwhom, type of attachment (if any), etc.

At 606, a content index is generated for the file. The content indexincludes information related to intrinsic properties of the file. Aspreviously described, the information included in the content index mayinclude adjectives and labels characterizing content of the document,described content of a video, identifying a tune, and so on.

At 608, the file is archived in a secondary storage. In variousimplementations, secondary storage may be located at a storage systemexternal to the user's computer and may include a tape drive, a diskarray, a cloud storage, etc.

At 610, the file attribute index 508 is stored at a first location, thecontent index 510 is stored at a second location and the archived copyof the file is stored at a third location (although such locations maybe different logical locations on the same storage device). Informationabout the first, second, and third locations, and associations of thearchived copy of the file with the corresponding index data are alsostored for use during the below described search process. The threelocations may be the same or different locations. For example, in someimplementations, the file attribute index 508 is generated and storedlocally at the user device 502, the content index 510 is generated at amedia agent 410 and stored locally on the media agent platform, and thearchived copy of the file is stored at a tape drive. In some otherimplementations, both the file attribute index 508 and the content index510 may be stored locally at the user device 502, e.g., to facilitatequick access to the stored information by a user and the archived copyis stored on a secondary storage device such as a cloud-based server.These implementations may be especially advantageous to wireless/ mobileuser devices 502 by storing all the information needed for browsing thearchived file at the user device 502, and making the archived fileavailable any time, and at any location, through the cloud-based archivestorage. In some implementations, the file attribute index 508 andcontent index 510 may be stored at the media agent platform and thearchive file may be stored at a backup storage system. These embodimentsmay be especially useful in corporate or managed networks by making thearchived file and the corresponding index information available to otherusers or system administrators even when the user device 502 is turnedoff or not connected to the managed network. Example data structuressuitable for storing the indexed information are described next.

Storing Multi-Dimensional Indexing Data

FIG. 7 depicts example data structures 700 suitable for storage ofindexing information discussed herein. File contents may be stored in adatabase as content word or phrase entries, and the files in which eachword entry occurs. For example, column 702 represents a listing ofvarious terms and phrases that were generated during content indexing offiles in a file archive. The column 704 may list each file within thefile archive for which the given term was generated during the indexing.The files corresponding to a given term entry may be ordered using arule such as the file having maximum occurrences of the term may belisted first. For each file in which a given term occurs, column 706 mayinclude a file location pointer indicative of where to locate the file.

The depicted example in FIG. 7 shows that the term “Accident” (entry708) was indexed during indexing of the file “crash.mov” (entry 710),the file “letter.docx” (entry 712) and the image file DFC0001.JPG (entry714). As an illustrative example, the term “accident” may have beengenerated during the content indexing of crash.mov based on theoccurrence of the word “accident” in the metadata description (orcritics review) associated with the file crash.mov. As an illustrativeexample, the word “accident” may have been generated during contentindexing of the file letter.docx because the text in the file mayinclude the term “accident.” As an illustrative embodiment, the word“accident” may be generated during context indexing of the imageDFC0001.JPG because the image is stored in a folder having the word“accident” in the folder name. Various other rules of associating fileswith search terms, not explicitly discussed in this document, are wellknown in the art.

The column 706 illustrates that file locators (boxes 716, 718, 720, 726)may be stored for each entry in the file index 704. The file locators716, 718, 720, 726 may be in the form of a universal resource locator(URL), or a filename (including the entire directory structure), or afilename and an offset within the file, and so on. In one aspect, thefile locators 716, 718, 720, 726 associated with a file index entry mayhave sufficient information to enable mounting to a drive such that thefile is viewable and can be interacted with the NFS 514 interface, aspreviously discussed. In some implementations, the file locators 716,718, 720, 726 may include number of data blocks (or bytes) within thefile archive where the indexed file (or portion of the file) can beretrieved from (or tape offsets and extents for files stored onsequential media like magnetic tape).

Content Indexing of Media Files

As further depicted in FIG. 7, another content term “zebra” (entry 722)may have been generated during indexing of a video file“c:/MyVideos/zoo1.wmv” (box 724). In some implementations, media filessuch as video files may be indexed both as a whole and in pieces by themulti-dimensional indexer 504. One dimension of indexing may simplyparse through metadata associated with the video file and index themetadata. The use of container tracks for providing information aboutvideo files is well known in the art. The metadata may be a part of thevideo file or may be stored in a separate file associated with the videofile.

For the above-discussed example of a media file index entry in box 724,the multi-dimensional indexer 504 may generate a clip-wise content indexto facilitate random access within the media file. For example,“zoo1.wmv” may be a home video of a user that may be a 30 minute videofile. However, a “zebra” may appear in the home video only from duration2 minutes 01 seconds to 2 minutes 36 seconds (box 728). Themulti-dimensional indexer may obtain the start-time, end-timeinformation using, e.g., a content pattern recognition engine or usingmetadata associated with the video file. The multi-dimensional indexer504 may include such start-time, end-time information in the file indexgenerated for a media file. The multi-dimensional indexer 504 mayfurther associate a list of pointers, links or URLs (box 732) that allowindependent access to portions of the video file when presenting thesearch results to the user, as further described below. One locationpointer in the list (e. g., the first URL) may contain locationinformation for a preview frame associated with the clip portion. Otherways to provide direct access to portions of a media file are possible,e.g., box 730, which shows indexing in the form of a start time (4minutes 30 seconds into the video) and a duration of the clip ofinterest (28 seconds), with an associated location pointer 734 to thebeginning of the clip or the preview frame for the clip.

Criteria-Based Search of the Search File System

FIG. 8 illustrates a search menu interface 800 presented to a user forbrowsing through archived files while looking for certain files thatmeet the user's search criteria. A search rules portion 802 may beprovided for the user to be able to specify search rules. The searchrules specified by the user may be used by the Search Filter 516 inselectively presenting files from the File Archive 512 that meet thesearch rules. The search menu interface 800 may include an action rulesportion 804 that allows a user to specify actions to be taken on filesthat are found as a result of the search. Note that the action rulesportion 804 is optional and may be eliminated in certain implementationsso that the action taken upon files found may simply depend on the filetypes, as is typically implemented in commonly used operating systems.

The search rules portion 802 may include content attribute menus 806 andfile attribute menus 808. The content attribute menus 806 may allow auser to specify search keywords, phrases, etc., as is well known in thefield of Internet searching. The file attribute menus 808 may allow auser to specify file characteristics, such as “modified before,”archived before,” a file type, file size constraints, and so on. Notethat while the content attribute menus 806 and file attribute menus 808are depicted as separate areas for the sake of illustration andexplanation, in some implementations, these two menus may be unified.For example, it is well known in the art of Internet searching thatsearch keywords can be terminated with special phrases such as“filetype:ppt” etc. to narrow the search results based on attributes offiles in which the search results are located.

The action rules portion 802 may includes several action options 810,812, 814, 816 etc. For example option 810 may be selected by a user toautomatically open a file (or files) upon search. The option 812selected by the user may specify that the search results may simply belisted, e.g., using the NFS interface 514. The option 814 may beselected by the user to indicate that a media file obtained as a resultof the search should be played upon finding. In some implementations, toavoid multiple simultaneous audio/video playbacks, this option may limitthe playback to a single search file—e.g., the first found file, or thefile that matches the search criteria the closest. The option 816 may beselected by the user to enable preview of one or more files found uponsearch. Various techniques for previewing files are possible, e.g., asdisclosed in the assignee's U.S. patent application No. 61/618,039, toLiu et al., having Attorney Docket No. 060692-8099.US00, incorporated byreference herein.

Presenting Search Results

FIG. 9 is a flow chart representation of a process 900 of presentingsearch results to a user, based on a filtered search of a file archive.The process 900 may be implemented on the user device 502, a media agentplatform or another computer.

At 902, a search interface (e.g., 800) comprising a search rules portionand an action rules portion is presented by the process 900 to a user.

At 904, a file system search criterion comprising at least one searchrule is received by the process 900. The search criterion may be, e.g.,based on user selection of menus using the search interface 800.

At 906, the process 900 searches a file archive using the searchcriterion (or criteria) and a set of files that are filtered using thesearch criterion (e.g., files that include the text phrase being searchfor) is generated. If no files matching the search criterion are found,an error message is generated. The error message may prompt the use toexpand his search or may simply report that no files were found. Aspreviously described, the search results may be made available to thedirectory search module on the user's device so that the search resultspresented to the user have the same look and feel as the nativedirectory browsing application and the user gets a seamless experiencein browsing search results.

At 908, an action (or actions) specified in the action rules portion maybe performed on the set of files generated above. In someimplementations, the action rules may be “implicit” as previouslydescribed, and the corresponding action may be triggered based on thetype of file found (e.g., a “hidden” file may be omitted from thedisplay, a picture file may be presented as a thumbnail, a video filemay be presented using a freeze frame, etc.).

Presenting Search Filtering Media Files

Multimedia file handling poses a special problem for the above-discussedsearch filtered file system because first, mounting of an entire mediafile, which may be several tens of Megabytes to Gigabytes in size) mayrequire a large amount of network bandwidth and computational resources.In addition, the mounting process may take an amount of time that someusers may find unacceptable. Furthermore, even when the resulting fileis mounted and made available for user interaction, e.g., through theNFS 514, a user may find it cumbersome to have to play back the mediafile; manually searching for what the user was looking for. For example,in the example previously discussed, when a user is looking for a“zebra” video, the video of interest may be only few seconds long in avideo file that may have a 30 minutes playback duration. In addition,some user devices 502 may not have sufficient memory or battery power tomount back a long media file locally and play back the entire file whilethe user looks for the content of interest. Even when a small clip ofthe content of interest is NFS mounted, a user may want to view videobefore or after the clip, which may not be locally available at a userdevice's primary storage, thereby leading to large delays whileadditional content is fetched from elsewhere.

The special media file handling disclosed herein is useful in addressingthe above-discussed operational issues, and other issues. In someimplementations, large video files are indexed as small video clips andcorresponding linked lists of pointers are stored. In one aspect, thisarrangement allows for random access to a smaller portion within thelarge media file. The Search Filter 516, e.g., may filter only therelevant video clips from a large media file and mount the clips fromthe file archive 512 to the directory listing of search filtered filesystem presented to the viewer. It will be appreciated by one of skillin the art that such mounting of a smaller portion of a file (e.g., a 2Megabyte portion of a 100 Megabyte file) is resource-efficient and mayprovide a much faster response time to the user's searches. Tofacilitate direct access to portions of a media file, in someimplementations, when archiving, the media file itself may be split intomultiple smaller portions that are stored as individual files.

When a user selects to play back the search results, (e.g., play option814), an appropriate application program (e.g., Windows Media Player orQuickTime player) may be invoked. A playback software agent executed onthe user device 502 may monitor the playback operation of a file fromthe search filtered file system. Depending on the current time ofplayback or where within the clip a user's current playback time is, theplayback software agent may speculatively fetch additional media datarepresenting content before or after play time of the content currentlymounted and being played. The playback software agent may similarly trapuser interactions with the application program (e.g., fast forward,rewind, pause) to appropriately fetch additional data from the filearchive, delete data from a local cache or neither delete nor fetchadditional data (e.g., when a user pauses the playback).

FIG. 10 is a flow chart representation of a process 1000 of playing backmedia files from a search filtered file system. In some implementations,the process 1000 is implemented on the user device 502.

At 1002, a playback request is received for archived content. Theplayback request may be issued by a user based on search resultspresented to the user, or automatically triggered based on anassociation between a known file type for a file found in the searchresults, as previously discussed.

At 1004, a portion of content is retrieved from a file archive forfulfilling the playback requested. The retrieval process may, e.g.,include reversing the compression, de-duplication, etc. performed on thecontent to present the content in its native (or original unarchived)format. The portion of content may correspond to, e.g., a time segmentof the content that was identified by the search filter.

At 1006, the process 1000 monitors the playback of the content. Theprocess 1000 monitors, e.g., the progress of the playback and how muchadditional data or time is remaining before the user device 502 runs outof content to be presented to the user. The process 1000 mayadditionally take into account the amount of time it takes to restore anext segment of content from the file archive.

Depending on the results of the monitoring, at 1008, additional mediacontent is transferred (by performing a similar reversing operation)from the archive to a local storage (e.g., play cache for the playback)or data may be deleted from the local storage, depending on the playbackoperation.

One of skill in the relevant art will appreciate that several techniqueshave been disclosed for providing a search filtered file system to auser using a file archive in a secondary storage. In one aspect, thesearch results are filtered to only show files that match the searchcriteria. In another aspect, media files are presented in smaller,independently accessible portions. It will further be appreciated thatthe disclosed techniques facilitate quick searching and previewing ofarchived content.

Conclusion

Systems and modules described herein may comprise software, firmware,hardware, or any combination(s) of software, firmware, or hardwaresuitable for the purposes described herein. Software and other modulesmay reside on servers, workstations, personal computers, computerizedtablets, PDAs, smart phones, and other devices suitable for the purposesdescribed herein. Modules described herein may be executed by ageneral-purpose computer, e.g., a server computer, wireless device, orpersonal computer. Those skilled in the relevant art will appreciatethat aspects of the invention can be practiced with othercommunications, data processing, or computer system configurations,including: Internet appliances, hand-held devices (including personaldigital assistants (PDAs)), wearable computers, all manner of cellularor mobile phones, multi-processor systems, microprocessor-based orprogrammable consumer electronics, set-top boxes, network PCs,mini-computers, mainframe computers, and the like. Indeed, the terms“computer,” “server,” “host,” “host system,” and the like, are generallyused interchangeably herein and refer to any of the above devices andsystems, as well as any data processor. Furthermore, aspects of theinvention can be embodied in a special purpose computer or dataprocessor that is specifically programmed, configured, or constructed toperform one or more of the computer-executable instructions explained indetail herein.

Software and other modules may be accessible via local memory, anetwork, a browser, or other application in an ASP context, or viaanother means suitable for the purposes described herein. Examples ofthe technology can also be practiced in distributed computingenvironments where tasks or modules are performed by remote processingdevices, which are linked through a communications network, such as aLocal Area Network (LAN), Wide Area Network (WAN), or the Internet. In adistributed computing environment, program modules may be located inboth local and remote memory storage devices. Data structures describedherein may comprise computer files, variables, programming arrays,programming structures, or any electronic information storage schemes ormethods, or any combinations thereof, suitable for the purposesdescribed herein. User interface elements described herein may compriseelements from graphical user interfaces, command line interfaces, andother interfaces suitable for the purposes described herein.

Examples of the technology may be stored or distributed oncomputer-readable media, including magnetically or optically readablecomputer disks, hard-wired or preprogrammed chips (e.g., EEPROMsemiconductor chips), nanotechnology memory, biological memory, or otherdata storage media. Indeed, computer-implemented instructions, datastructures, screen displays, and other data under aspects of theinvention may be distributed over the Internet or over other networks(including wireless networks), on a propagated signal on a propagationmedium (e.g., an electromagnetic wave(s), a sound wave, etc.) over aperiod of time, or they may be provided on any analog or digital network(packet switched, circuit switched, or other scheme).

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” As used herein, the terms “connected,”“coupled,” or any variant thereof, means any connection or coupling,either direct or indirect, between two or more elements; the coupling orconnection between the elements can be physical, logical, or acombination thereof. Additionally, the words “herein,” “above,” “below,”and words of similar import, when used in this application, refer tothis application as a whole and not to any particular portions of thisapplication. Where the context permits, words in the above DetailedDescription using the singular or plural number may also include theplural or singular number respectively. The word “or,” in reference to alist of two or more items, covers all of the following interpretationsof the word: any of the items in the list, all of the items in the list,and any combination of the items in the list.

The above Detailed Description is not intended to be exhaustive or tolimit the invention to the precise form disclosed above. While specificexamples for the invention are described above for illustrativepurposes, various equivalent modifications are possible within the scopeof the invention, as those skilled in the relevant art will recognize.For example, while processes or blocks are presented in a given order,alternative implementations may perform routines having steps, or employsystems having blocks, in a different order, and some processes orblocks may be deleted, moved, added, subdivided, combined, and/ormodified to provide alternative or subcombinations. Each of theseprocesses or blocks may be implemented in a variety of different ways.Also, while processes or blocks are at times shown as being performed inseries, these processes or blocks may instead be performed orimplemented in parallel, or may be performed at different times. Furtherany specific numbers noted herein are only examples: alternativeimplementations may employ differing values or ranges.

The teachings of the invention provided herein can be applied to othersystems, not necessarily the systems described herein. The elements andacts of the various examples described above can be combined to providefurther implementations of the invention.

Any patents and applications and other references noted above, includingany that may be listed in accompanying filing papers, are incorporatedherein by reference. Aspects of the invention can be modified, ifnecessary, to employ the systems, functions, and concepts of the variousreferences described above to provide yet further implementations of theinvention.

These and other changes can be made to the invention in light of theabove Detailed Description. While the above description describescertain examples of the invention and describes the best modecontemplated, no matter how detailed the above appears in text, theinvention can be practiced in many ways. Details of the system may varyconsiderably in its specific implementation, while still beingencompassed by the invention disclosed herein. As noted above,particular terminology used when describing certain features or aspectsof the invention should not be taken to imply that the terminology isbeing redefined herein to be restricted to any specific characteristics,features, or aspects of the invention with which that terminology isassociated. In general, the terms used in the following claims shouldnot be construed to limit the invention to the specific examplesdisclosed in the specification, unless the above Detailed Descriptionsection explicitly defines such terms. Accordingly, the actual scope ofthe invention encompasses not only the disclosed examples, but also allequivalent ways of practicing or implementing the invention under theclaims.

While certain examples are presented below in certain forms, theapplicant contemplates the various aspects of the invention in anynumber of claim forms. Accordingly, the applicant reserves the right toadd additional claims after filing the application to pursue suchadditional claim forms for other aspects of the invention.

1. A method of archiving a file to secondary storage in an informationmanagement system, the method comprising: archiving a file to secondarystorage, by a media agent component of the information management systemthat has been identified for archiving the file, wherein the archivingcomprises: (a) dividing the file being archived into a plurality ofindependently accessible smaller portions of an archive copy of thefile, and (b) multi-dimensional indexing of the file being archived,wherein the multi-dimensional indexing comprises: (i) generating a fileattribute index, which includes extrinsic information about the filebeing archived that is independent of file contents, and (ii) generatinga content index of at least one of: text content, picture content, videocontent, and audio content of the file being archived, and (c) whereineach independently accessible smaller portion in the plurality ofindependently accessible smaller portions of the archive copy isidentified at least once in at least one of the file attribute index andthe content index; providing to a user of the information managementsystem a fast-forward progression through the plurality of independentlyaccessible smaller portions of the archive copy without restoring eachindependently accessible smaller portion from secondary storage, whereinthe fast-forward progression is based on searching at least one of thefile attribute index and the content index; and restoring a firstindependently accessible portion of the archive copy to a user-devicecomponent of the information management system without having to alsorestore the other independently accessible smaller portions of thearchive copy, wherein the first independently accessible smaller portionof the archive copy is found by a search filter searching at least oneof the file attribute index and the content index within the informationmanagement system.
 2. The method of claim 1 further comprising:providing a user of the information management system with a searchutility coupled to the search filter, wherein the search utility isbased on multi-dimensional indexing performed in the course of archivinga plurality of files in the information management system; and restoringfrom secondary storage only those archive copies or independentlyaccessible smaller portions thereof that satisfy the user's searchcriteria, based on a respective file attribute index and content indexassociated with each archive copy.
 3. The method of claim 1, wherein thefile attribute index and the content index are located on theuser-device component of the information management system; and whereinthe archive copy is located on a secondary storage component which isdistinct from the user-device component.
 4. The method of claim 1,wherein generating the content index comprises pattern recognition thatextracts content information about one or more images in the file beingarchived.
 5. The method of claim 1, wherein generating the content indexcomprises pattern recognition that extracts content information aboutone or more audio sequences in the file being archived.
 6. The method ofclaim 1, wherein generating the content index comprises patternrecognition that extracts content information about one or more videosequences in the file being archived.
 7. The method of claim 1, whereinthe multi-dimensional indexing comprises generating pointers to thelocations in the secondary storage of each respective independentlyaccessible smaller portion of the archive copy.
 8. The method of claim 1further comprising: storing the content index to a first location wherethe content index may be searched by the search filter, wherein thefirst location is in the primary storage; and storing the file attributeindex to a second location where the file attribute index may besearched by the search filter, wherein the second location is also inthe primary storage.
 9. The method of claim 1 further comprising:storing the content index to a first location where the content indexmay be searched by the search filter, wherein the first location isassociated with the media agent; and storing the file attribute index toa second location where the file attribute index may be searched by thesearch filter, wherein the second location is also associated with themedia agent.
 10. A method comprising: multi-dimensional indexing of afile being archived to secondary storage in an information managementsystem, by a multi-dimensional indexer, wherein the file being archivedis stored in primary storage which is in communication with a usercomputing device, and wherein the multi-dimensional indexing comprises:(a) generating a file attribute index, which includes extrinsicinformation about the file being archived that is independent ofcontents of the file, and (b) generating a content index based on atleast one of: text content, picture content, video content, and audiocontent of the file being archived; archiving the file to secondarystorage, by a media agent component of the information managementsystem, wherein the file archiving includes dividing the file into aplurality of independently accessible smaller portions of an archivecopy of the file being archived, and wherein each independentlyaccessible smaller portion in the plurality of independently accessiblesmaller portions of the archive copy is identified at least once in atleast one of the file attribute index and the content index; providingto a user of the information management system a fast-forwardprogression through the plurality of the independently accessiblesmaller portions of the archive copy without restoring from thesecondary storage each independently accessible smaller portion, whereinthe fast-forward progression is based on searching at least one of thefile attribute index and the content index; restoring a firstindependently accessible smaller portion of the archive copy to theprimary storage in communication with the user computing device, withoutalso restoring from the secondary storage the other independentlyaccessible smaller portions in the plurality of independently accessiblesmaller portions of the archive copy; and wherein the multi-dimensionalindexer executes as computer-executable instructions on a hardwareprocessor coupled to non-transitory computer memory storing theinstructions.
 11. The method of claim 10 wherein the hardware processoris part of the user computing device that is in communication with theprimary storage.
 12. The method of claim 10 wherein the hardwareprocessor is part of a secondary storage computing device that executesthe media agent component of the information management system.
 13. Themethod of claim 10 further comprising: accessing, by the user computingdevice, the restored first independently accessible smaller portion ofthe archive copy in the primary storage.
 14. The method of claim 10,wherein the restoring is performed by a media agent component of theinformation management system.
 15. The method of claim 10, wherein therestoring is performed by a media agent component of the informationmanagement system in response to an indication received from the usercomputing device searching at least one of the file attribute index andthe content index.
 16. The method of claim 10 wherein the file beingarchived is a media file, and wherein the restored first independentlyaccessible smaller portion of the archive copy comprises a picture thatsatisfies a search criterion.
 17. The method of claim 10 wherein thefile being archived is a media file, and wherein the restored firstindependently accessible smaller portion of the archive copy comprisesan audio segment that satisfies a search criterion.
 18. The method ofclaim 10 wherein the file being archived is a media file, and whereinthe restored first independently accessible smaller portion of thearchive copy comprises a video segment that satisfies a searchcriterion.
 19. The method of claim 10 further comprising: providing theuser of the information management system with a search utility forsearching the secondary storage based on the multi-dimensional indexing;and wherein the restoring comprises mounting to the primary storage onlysuch independently accessible smaller portions of the archive copy thatsatisfy search criteria supplied by the user and applied by the searchutility.
 20. A computer-readable storage medium, excluding transitorypropagating signals, storing instructions that, when executed by aprocessor, cause the processor to execute a method of enabling a user tosearch from a user-device for content stored in a file archive of aninformation management system, based on multi-dimensional indexing of aplurality of files archived to the file archive, the method comprising:providing a search interface; receiving a search criterion; searching,based on the search criterion, indexing information about the filearchive, which comprises respective archive copies of the plurality offiles archived to the file archive, wherein the indexing informationcomprises, for each respective archive copy in the file archive: (i) afile attribute index generated by a media agent when the respective filewas archived to the file archive, and (ii) a content index generated byat least one of the media agent and a user-device when the respectivefile was archived to the file archive, wherein the content index indexesat least part of the contents of the file being archived based on atleast one of: text content, picture content, video content, and audiocontent, and wherein the search criterion indicates at least one of:text content, picture content, video content, and audio content to besearched for; and performing an action on a first archive copy in thefile archive, which archive copy satisfies the search criterion used inthe searching, wherein the action comprises providing a fast-forwardprogression through a plurality of smaller portions of the archive copywithout individually restoring the smaller portions of the archive copy,wherein the fast-forward progression is based on searching at least oneof the file attribute index and the content index of the archive copy,and wherein each of the smaller portions of the archive copy is indexedat least once, during the archiving of the file to the file archive, inat least one of the file attribute index and the content index of thearchive copy, and may be restored to the user-device without the othersmaller portions of the archive copy also being restored.
 21. Thecomputer-readable storage medium of claim 20 wherein the method furthercomprises: mounting the archive copy that satisfies the search criterionused in the searching to a drive accessible by the user-device using anetwork file system (NFS) interface without also mounting archive copiesthat do not satisfy the search criterion.
 22. The computer-readablestorage medium of claim 20 wherein the method further comprises:mounting the archive copy that satisfies the search criterion used inthe searching to a drive accessible by the user-device using a networkfile system (NFS) interface without also mounting archive copies that donot satisfy the search criterion; and restoring to the user-device afirst smaller portion of the archive copy without also restoring theother smaller portions of the archive copy.
 23. The computer-readablestorage medium of claim 20 wherein the method further comprises:mounting the archive copy that satisfies the search criterion used inthe searching to a drive accessible by the user-device using a networkfile system (NFS) interface without also mounting archive copies that donot satisfy the search criterion; restoring to the user-device a firstsmaller portion of the archive copy in response to a user commandindicating the first smaller portion; and restoring to the user-device,without receiving a further user command, a second smaller portion ofthe archive copy which, according to the content index, is immediatelybefore or immediately after the first smaller portion of the archivecopy in the fast-forward progression; and wherein the first and secondsmaller portions are restored without also restoring the other smallerportions of the archive copy.